The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

Diffusion models, which learn to reverse a signal destruction process to generate new data, typically require the signal at each step to have the same dimension. We argue that, considering the spatial redundancy in image signals, there is no need to maintain a high dimensionality in the evolution process, especially in the early generation phase. To this end, we make a theoretical generalization of the forward diffusion process via signal decomposition. Concretely, we manage to decompose an image into multiple orthogonal components and control the attenuation of each component when perturbing the image. That way, along with the noise strength increasing, we are able to diminish those inconsequential components and thus use a lower-dimensional signal to represent the source, barely losing information. Such a reformulation allows to vary dimensions in both training and inference of diffusion models. Extensive experiments on a range of datasets suggest that our approach substantially reduces the computational cost and achieves on-par or even better synthesis performance compared to baseline methods. We also show that our strategy facilitates high-resolution image synthesis and improves FID of diffusion model trained on FFHQ at $1024\times1024$ resolution from 52.40 to 10.46. Code and models will be made publicly available.

对于在开放世界中部署的机器学习模型是必不可少的。最近，在训练期间（也称为离群暴露）在训练期间使用辅助外离群值数据集已显示出令人鼓舞的性能。由于潜在的OOD数据的样本空间可能是过大的，因此进行抽样信息的异常值至关重要。在这项工作中，我们提出了一种新型的基于后取样的离群矿井诗歌诗，该诗歌有助于有效利用异常数据，并促进了ID和OOD数据之间的紧凑决策边界，以改善检测。我们表明，诗在普通基准上建立了最先进的表现。与当前使用贪婪采样策略的最佳方法相比，诗在CIFAR-10和CIFAR-100上分别提高了相对性能的42.0％和24.2％（FPR95）。我们进一步提供了有关诗歌检测有效性的理论见解。

深度图用于从3D渲染到2D图像效应（例如散景）的广泛应用。但是，单个图像深度估计（侧）模型预测的人通常无法捕获对象中的孤立孔和/或具有不准确的边界区域。同时，使用商业自动掩蔽工具或现成的分割和垫子的方法，甚至是通过手动编辑，使用商业自动掩盖工具或现成的方法更容易获得。因此，在本文中，我们提出了一个新的掩盖引导深度细化的问题，该问题利用通用掩模来完善侧面模型的深度预测。我们的框架执行了分层的细化和介入/架设，将深度图分解为两个由掩码和倒置面罩表示的单独的层。由于具有深度和掩码注释的数据集很少，因此我们提出了一种使用任意掩码和RGB-D数据集的自我监督学习方案。我们从经验上表明，我们的方法对不同类型的掩模和初始深度预测具有鲁棒性，可以准确地完善内部和外掩模边界区域的深度值。我们通过消融研究进一步分析了我们的模型，并证明了实际应用的结果。可以在https://sooyekim.github.io/maskdepth/上找到更多信息。

近年来，多智能体加固学习（Marl）在各种应用中呈现出令人印象深刻的性能。但是，物理限制，预算限制以及许多其他因素通常会在多代理系统（MAS）上施加\ Texit {约束}，这不能由传统的Marl框架处理。具体而言，本文重点介绍受约束的Mase，其中代理工作\纺织{合作}在各种限制下最大化预期的团队平均成本下的预期团队平均返回，并开发一个名为DECOM的\ TEXTIT {约束合作MARL}框架，名为DECOM这样的苗条。特别是，DECOM将每个代理人的策略分解为两个模块，这使得代理商之间的信息共享，以实现更好的合作。此外，通过这种模块化，DREM的训练算法将原始约束优化分为奖励的无约束优化和成本的约束满足问题。然后，Decom以计算有效的方式迭代地解决这些问题，这使得DECOM高度可扩展。我们还提供了对Decom策略更新算法的融合的理论保障。最后，我们在玩具和大规模（有500个代理）环境中使用各种类型的成本验证了DECOM的有效性。

平衡勘探和剥削对加强学习（RL）至关重要。在本文中，我们在理论上和经验上，研究了用于连续状态行动空间的加固学习（PSRL）的模型后采样。首先，我们在连续空间中显示PSRL的第一个遗憾，这是我们知识中的最佳地段中的多项式。假设奖励和转换函数可以由贝叶斯线性回归建模，我们开发了$ \ tilde {o}的遗憾（h ^ {3/2} d \ sqrt {t}）$，其中$ h $剧集长度，$ D $是状态动作空间的维度，$ t $表示总时间步骤。此结果与线性MDP中的非PSRL方法的最佳已知的遗憾符合。我们的绑定可以扩展到非线性情况以及功能嵌入功能：在特征表示上的线性内核$ \ phi $，后悔绑定成为$ \ tilde {o}（h ^ {3/2} d _ {\ phi} \ SQRT {T}）$，其中$ d_ \ phi $是表示空间的尺寸。此外，我们呈现MPC-PSRL，一种基于模型的后部采样算法，具有用于动作选择的模型预测控制。为了捕获模型中的不确定性，我们在神经网络的倒数第二层（特征表示层$ \ phi $）上使用贝叶斯线性回归。实证结果表明，与基于模型的算法相比，我们的算法在基准连续控制任务中实现了最先进的示例效率，并匹配无模型算法的渐近性能。

Natural Language Processing (NLP) has been revolutionized by the use of Pre-trained Language Models (PLMs) such as BERT. Despite setting new records in nearly every NLP task, PLMs still face a number of challenges including poor interpretability, weak reasoning capability, and the need for a lot of expensive annotated data when applied to downstream tasks. By integrating external knowledge into PLMs, \textit{\underline{K}nowledge-\underline{E}nhanced \underline{P}re-trained \underline{L}anguage \underline{M}odels} (KEPLMs) have the potential to overcome the above-mentioned limitations. In this paper, we examine KEPLMs systematically through a series of studies. Specifically, we outline the common types and different formats of knowledge to be integrated into KEPLMs, detail the existing methods for building and evaluating KEPLMS, present the applications of KEPLMs in downstream tasks, and discuss the future research directions. Researchers will benefit from this survey by gaining a quick and comprehensive overview of the latest developments in this field.

We investigate response generation for multi-turn dialogue in generative-based chatbots. Existing generative models based on RNNs (Recurrent Neural Networks) usually employ the last hidden state to summarize the sequences, which makes models unable to capture the subtle variability observed in different dialogues and cannot distinguish the differences between dialogues that are similar in composition. In this paper, we propose a Pseudo-Variational Gated Recurrent Unit (PVGRU) component without posterior knowledge through introducing a recurrent summarizing variable into the GRU, which can aggregate the accumulated distribution variations of subsequences. PVGRU can perceive the subtle semantic variability through summarizing variables that are optimized by the devised distribution consistency and reconstruction objectives. In addition, we build a Pseudo-Variational Hierarchical Dialogue (PVHD) model based on PVGRU. Experimental results demonstrate that PVGRU can broadly improve the diversity and relevance of responses on two benchmark datasets.

Despite impressive success in many tasks, deep learning models are shown to rely on spurious features, which will catastrophically fail when generalized to out-of-distribution (OOD) data. Invariant Risk Minimization (IRM) is proposed to alleviate this issue by extracting domain-invariant features for OOD generalization. Nevertheless, recent work shows that IRM is only effective for a certain type of distribution shift (e.g., correlation shift) while it fails for other cases (e.g., diversity shift). Meanwhile, another thread of method, Adversarial Training (AT), has shown better domain transfer performance, suggesting that it has the potential to be an effective candidate for extracting domain-invariant features. This paper investigates this possibility by exploring the similarity between the IRM and AT objectives. Inspired by this connection, we propose Domainwise Adversarial Training (DAT), an AT-inspired method for alleviating distribution shift by domain-specific perturbations. Extensive experiments show that our proposed DAT can effectively remove domain-varying features and improve OOD generalization under both correlation shift and diversity shift.

In many applications, heterogeneous treatment effects on a censored response variable are of primary interest, and it is natural to evaluate the effects at different quantiles (e.g., median). The large number of potential effect modifiers, the unknown structure of the treatment effects, and the presence of right censoring pose significant challenges. In this paper, we develop a hybrid forest approach called Hybrid Censored Quantile Regression Forest (HCQRF) to assess the heterogeneous effects varying with high-dimensional variables. The hybrid estimation approach takes advantage of the random forests and the censored quantile regression. We propose a doubly-weighted estimation procedure that consists of a redistribution-of-mass weight to handle censoring and an adaptive nearest neighbor weight derived from the forest to handle high-dimensional effect functions. We propose a variable importance decomposition to measure the impact of a variable on the treatment effect function. Extensive simulation studies demonstrate the efficacy and stability of HCQRF. The result of the simulation study also convinces us of the effectiveness of the variable importance decomposition. We apply HCQRF to a clinical trial of colorectal cancer. We achieve insightful estimations of the treatment effect and meaningful variable importance results. The result of the variable importance also confirms the necessity of the decomposition.